Experiments with Cholesky Factorization on Clusters of SMPs

Cholesky factorization of large dense matrices is an integral part of many applications in science and engineering. In this paper we report on experiments with different parallel versions of Cholesky factorization on modern high-performance computing architectures. For the parallelization of Cholesky factorization we utilized various standard linear algebra software packages and present performance results on SMP clusters and shared-memory cc-NUMA machines. Clusters of SMPs can be characterized as hybrid parallel architectures which combine the main architectural features of distributed-memory and shared-memory parallel computers. Although the availability of SMP clusters is increasing rapidly within the scientific computing community, currently no generally accepted programming model exists for these machines. As a consequence, most application developers utilize pure distributed-memory programming models, usually based on the message passing interface (MPI), and thus may miss a number of optimization opportunities offered by the shared-memory available within the nodes of a cluster. In order to address these issues, we have experimented with different parallelization strategies for Cholesky decomposition comparing pure message passing strategies to a hybrid parallelization strategy that combines message passing with shared-memory parallelization based on multi-threading.